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ABSTRACT 



This report presents findings from an evaluation of the 
National Science Foundation’s (NSF’s) Urban Systemic Initiative (USI) program 
in 22 large urban school districts. The USI program has supported educational 
reform in large public school districts that have a high percentage of 
students living in poverty and has directed its efforts toward the 
improvement of mathematics and science achievement in these schools. The 
NSF's Six Drivers of Systemic Reform provided a framework for USI 
implementation, focusing on standards-based curriculum and instruction, 
aligned assessment, policies, professional development, convergence of 
resources, leadership, and partnerships. The report presents evidence of 
noteworthy gains in student achievement, with the greatest gains seen in 
districts participating in the USI program for the longest period of time. 
Urban students in USI districts have substantially increased their enrollment 
rates in mathematics and science gatekeeping and higher level courses. 
Under-represented minority students made even greater enrollment gains than 
their peers during the same time period, resulting in reduced enrollment 
disparities. More 11th and 12th graders are taking college entrance 
examinations, indicating that more students aspire to pursue higher 
education. These advances are accompanied by evidence that urban districts 
are developing the infrastructure to sustain achievement gains (policies 
encouraging enrollment in gatekeeping and higher level mathematics and 
science courses, strengthened professional development, new ways of managing 
partnerships and resources, and data-driven accountability systems) . (SM) 
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E^ECUTOWE SUMMARY 



Since 1993, the National Science Foundation’s Urban Systemic Initiative (USI) program 
has been a catalyst for large-scale systemic change directed towards improving the 
science and mathematics achievement of ALL urban students. This report presents 
preliminary findings from an evaluative study of NSF’s USI program among 22 large 
urban school districts. NSF’s Six Drivers of Systemic Reform provided a framework for 
USI implementation, focusing on standards-based curriculum and instruction, aligned 
assessment, policies, professional development, convergence of resources, leadership, 
and partnerships. 

This report presents evidence of noteworthy gains in student achievement, with the 
greatest gains seen in school districts that have participated in the USI program for the 
longest period of time. Urban students in USI school districts have substantially 
increased their enrollment rates in mathematics and science gate-keeping and higher- 
level courses. Underrepresented minority students made even greater enrollment gains 
than their peers during the same period, resulting in reduced enrollment disparities. 
Assessment test results show that USI students have made gains in science and 
mathematics achievement, while reducing achievement gaps among racial/ethnic groups. 
The increasing numbers of 1 1th and 12th grade students taking college entrance 
examinations (AP, SAT, and ACT) indicate that more students have aspirations of 
pursuing post-secondary education. 

These advances are accompanied by evidence that urban districts are developing the 
infrastructure to sustain achievement gains — policies that encourage enrollment in gate- 
keeping and higher-level mathematics and science courses, strengthened professional 
development programs, new ways of managing partnerships and resources, and data- 
driven accountability systems. USI programs have invested heavily in professional 
development believing it to be a key lever for improving student outcomes. Our Enacted 
Curriculum Survey found that teachers with professional development in standards- 
based curriculum and instruction report teaching practices that are more consistent with 
state and national standards. 

The infrastructure of partnerships built around each USI site provided strong support 
for systemic changes based on their unique capacity, resources, and experiences. The 
partners and school districts interacted as part of a unified effort to promote and support 
large-scale education reform. Partnerships were formed with interested community 
members including higher education institutions, corporations, foundations, informal 
institutions, research centers and laboratories, parents, and other federal/state/local 
programs. 

Education reform is a complex, long-term process that requires simultaneous changes 
in expectations, policies, curriculum, assessment, professional development, student 
support systems, data use, and the allocation of resources. As students are exposed to 
high expectations, a rigorous and engaging curriculum, and better instructional 
methods, NSF’s investment will continue to result in improved outcomes for all urban 
students. Much has been done; more remains to be accomplished. 
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ACCOMPLISHMENTS OF UlTOAIM SCHOOL STUOEWTS 

Preliminary findings from the ongoing “How Reform Works: An Evaluative Study of 
National Science Foundation’s Urban Systemic Initiatives” indicate significant 
improvement in mathematics and science achievement among K-12 students in major 
urban school districts. Mathematics and science gate-keeping course enrollment and 
completion rates have steadily increased since school year (SY) 93-94. Many of the 
assessment test results also indicate steady improvement. In the majority of the sites, 
underrepresented minority students made greater academic gains in mathematics 
and science than their peers during the same time period, which resulted in reduced 
achievement gaps. This report summarizes achievements in 21 USI sites in 
Cohorts 93 - 97. Cohort 99 (Houston) is not represented as no trend data were available 
by SY 98-99. 



UR1BAIM SYSTEMS 0W0T0AT0WES (OSD) 

To enhance the academic achievement of urban 
students, the USI program has supported educational 
reform in 22 large public school districts that have a 
high percentage of their students living in poverty. 

The aim is to improve the mathematical and scientific 
literacy of ALL students in urban communities. 

A solid foundation in mathematics and science will 
prepare students for higher education in these areas, 
and allow them to pursue 21st century careers in 
mathematics, science, and technology. 

NSF launched the USI program in 1993, applying 
lessons learned from its earlier reform effort, the State 
Systemic Initiative (SSI). Other NSF educational 
reform programs include the Rural Systemic Initiative 
(RSI), and, for middle-sized cities, the Comprehensive 
Partnerships for Mathematics and Science 
Achievement (CPMSA). The USI and CPMSA 
systemic initiative programs have recently evolved into 
the Urban Systemic Program (USP). 



22 USI Sites 



Cohort 93 

Baltimore, Chicago, 
Dallas, Detroit, 

El Paso, Miami-Dade, 
New York, Phoenix 



Cohort 94 

Cleveland, Columbus, 
Fresno, Los Angeles, 
Memphis, New Orleans, 
Philadelphia 



Cohort 95 

Milwaukee, St. Louis, 
San Antonio, San Diego 



Cohort 97 

Atlanta, Jacksonville 



Cohort 99 

Houston 



Urban Systemic Initiatives 




Cohort 93 (8) 
Cohort 94 (7) 
Cohort 95 (4) 
Cohort 97 (2) 
Cohort 99(1) 



Total (22 Sites) 
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These NSF programs are catalysts for large-scale systemic changes directed toward 
improving the mathematics and science achievements of all students, including those 
historically underserved. NSF stipulates that all children can learn if they are provided 
with rich learning environments. The goal is comprehensive systemwide changes that 
dynamically create and sustain an educational infrastructure to support mathematics 
and science teaching and learning. This infrastructure includes standards-based 
curriculum and instruction, aligned assessment, professional development for teachers, 
convergence of educational resources, and community partnerships. Additional 
crosscutting issues are equity, quality, scaling up, coordination, and organization. 



USI Facts in Brief (SY 1998-99) 

Students: Total 4.5 Million 

Black 41% 

Hispanic 38% 

White 1 5% 

Asian/Pacific Islander 6% 
American Indian 0.3% 

Male 51 % , Female 49% 



Schools: Total 5,559 

3,710 Elementary Schools 
837 Middle Schools 
864 High Schools 
148 Ungraded Schools 

Teachers: Total 158,163 

126,018 Elementary 

17,143 Mathematics G6-12 
15,002 Science G6-12 

75% of teachers participated in Professional Development 

Selected Key Indicators 
(Average of USI sites) 

Free/Reduced Lunch 69% 

Limited English Proficient 12% 

Special Education 13% 

Daily Attendance 90% 

Drop-out Rate 12% 

Mobility Rate 31 % 

Per Pupil Spending $7,278 
Average Class Size 24.4 
Classrooms with Internet Access 53% 



Sources: 

• Urban School Key Indicators of Science and Mathematics Education - 
Based on Key Indicator Data System (KIDS- 1999), Volume IV, 

Systemic Research, Inc., September 2000. 

• Systemic Initiatives Core Data Elements Findings: 1998-1999 School Year 
Collection Summary Report (CDE-1999), Westat, Inc. and QRC, July 2000. 
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D IMPACT ®F OSD 



When the USI began, the program impacted about 2.2 million students in the first 
cohort of eight USI sites as shown in Exhibit I. Between 1994 and 1997 three additional 
cohorts received USI grants bringing the total to 21 cities. In SY 98-99 the number of 
students in these cities was over 4.5 million, or 10% of all American students who 
attended public schools. 

About 80% of those attending these urban schools are underrepresented minority 
students. They are also among the poorest students in the United States, with 69% of 
the students eligible to receive free or reduced lunch. In addition, 12% are children 
who have limited English proficiency and 13% receive special education services. 

During SY 98-99, over 5,500 schools were impacted by the USI program. About 86% of 
these schools received direct services from the program. 

In SY 98-99, USI districts included 158,163 teachers: 126,018 elementary, 17,143 
secondary mathematics, and 15,002 secondary science teachers. During that school year 
75% (1 18,399) of these teachers participated in professional development activities. 

Exhibit I: Number of Students Impacted by USI 

Unit 1,000 
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“In the 5 years I have been here 3 we are more into active learning with students. USI has 
had a major impact on it. We have gone from a school of teachers lecturing to teachers 
becoming more the guides and students working cooperatively. You see them up and moving 
and involved. 33 



- A Principal in a USI school 



§m DROVERS OF SYSTE1M10C REFORM 

Over the course of its systemic initiative programs, NSF has developed a theoretical 
structure for systemic reform. There are six “drivers” that NSF has found to be 
central to successful school reform. Our preliminary study findings indicate that 
implementation of these drivers has an important influence in successfully reforming 
and restructuring school district infrastructure within each city. 



NSF's Six Drivers of Systemic Reform 

1. Standard-based Curriculum, Instruction, and Assessment 

2. Policy Support for High Quality Learning and Teaching 

3. Convergence of Educational Resources 

4. Partnerships and Leadership: Broad-Based Support 

5. Measures of Effectiveness Focused on Student Outcomes 

6. Achievement of ALL Students, Including Those Historically Underserved 



STAWDARD-BASEO CURROCULUM AMD BIWSTRUCTOOW 

Standards-based curriculum means that instructional methods and materials are aligned 
with national and/or state standards. The purpose of the standards is to define what all 
children should know and be able to do at certain points in their education. 

There are many national and state standards for mathematics and science. Many USI 
sites align their mathematics curriculum to the National Council of Teachers of 
Mathematics (NCTM) Standards, and their science curriculum to the American 
Association for the Advancement of Science (AAAS) Benchmarks, National Research 
Council (NRC) Standards, or to the National Science Education Standards (NSES). 

Numerous states have developed their own standards that are aligned with the national 
standards. For example, Dallas, El Paso, and San Antonio abide by the Texas Essential 
Knowledge and Skills standards, and Miami-Dade and Jacksonville follow the Florida 
State Sunshine Standards. 

Nearly all of the USI sites have decreased their emphasis on lecture and demonstration 
methods of teaching and moved towards inquiry-based instruction. Some educational 
theorists - Dewey, Brunner, Piaget, and Featherstone - believe that the hands-on, 
inquiry-based method of instruction benefits most students’ learning. Students learn 
when they actively work out solutions to problems themselves. 

Curriculum materials based on national and state standards and the inquiry-based mode 
of instruction are widely available. Numerous USI sites have adopted many of the same 
materials - typical examples are as follows: 



Mathematics 


Science 


• Connected Math Project (CMP) G6-8 

• Everyday Mathematics K-G6 

• Interactive Mathematics Program 
(IMP) G9-1 2 

• Investigations in Number, Data, and 
Space K-G5 

• Mathland K-G6 


• Full Option Science System (FOSS) K-G6 

• Insights K-G6 

• Science and Technology for Children (STC) 
K-G6 

• Science Education for Public Understanding 
Program (SEPUP) G4-9 
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The USI program, as do all Federal programs, must follow the Government Performance 
and Results Act (GPRA). For the USI, GPRA requires implementation of a standards- 
based curriculum by 80% of the schools that participated in the program for at least 
three years. A school is in compliance if at least one third of its mathematics and science 
teachers have implemented a standards-based curriculum. According to the Core 
Data Elements (NSF’s annual data collection system) for SY 98-99, 86.8% of USI schools 
reached the GPRA goal for mathematics and 86.6% reached the goal for science as 
shown in Exhibit II. 

Exhibit II: USI Schools Implementing Standards-Based Curriculum* 



Mathematics 



Science 



80 % 

60 % 

40 % 

20 % 



89 % 



90 % 



( 1 , 615 ) 



( 449 ) 



( 344 ) 



86 . 8 % 



( 2 , 443 ) 



Elementary Middle High 



100 % 

80 % 

60 % 

40 % 

20% 

0 



89 % 



( 1 , 606 ) 



86% 



( 429 ) 



( 369 ) 



86 . 6 % 

( 2 , 440 ) 



Elementary Middle High 



* Percent of schools participating for 3 years or more in which at least one-third of the teachers teaching 
mathematics and science have implemented a standards-based curriculum. 

(# ) indicates number of schools 
Source: CDE-1999 



SmWOAROS-IBASEO ASSESSIMIEMT 

Assessment provides critical information about an individual student’s mathematics and 
science achievement. Assessment also provides a feedback system that allows an entire 
school district to account to the public about progress in helping all children to learn. 

In order to meet these ends, testing instruments must be aligned to both curriculum 
content and instructional methods. 

Several trends emerged from our analysis of assessment practices. Standardized 
assessment is more common in elementary and middle schools than it is in high schools. 
In addition, more school districts assess mathematics achievement than science 
achievement. 



Assessment Instrument 


Number of USI Si 
Standardi 

Mathematics 

Elem Middle High 


tes Implementing 
ized Tests 

Science 

Elem Middle High 


District Designed Test 


1 


1 


2 


1 


1 


2 


Iowa Test of Basic Skills 


4 


4 


2 


3 


3 


2 


Standford Achievement Test 8/9 


8 


8 


6 


7 


7 


4 


State Designed Test 


11 


12 


10 


7 


11 


9 


Other 


3 


2 


2 


4 


4 


3 



POUCY SUPPORT FOR HDGWI QUALBTY LEAR^DWG AMD TEACHDMG 

One challenge for urban school districts is to establish policies that create the 
expectation that all students can reach the same high standards, while being responsive 
to different learning styles and being sensitive to the needs of a diverse group of 
children. Policies that promote equal access to high quality, challenging mathematics 
and science courses in high school are the backbone of mathematics and science reform. 
Gate-keeping and higher-level mathematics and science courses provide a solid 
foundation in these subjects. Research has indicated that students who enroll in and 
complete these courses are more likely to pursue higher-level courses and post-secondary 
education. These courses are as follows: 



Mathematics 


Science 


• Algebra 1 or above in 8th Grade 


• Biology 


• Geometry 


• Chemistry 


• Algebra II 


• Physics 


• Calculus 


• Integrated Science 1, II, and III 


• Integrated Math 1, II, III, and IV 





Of the many policies and practices that have a potential impact on student achievement, 
many USI sites have implemented the following policies promoting access to high quality 
learning and teaching: 

° Scale-up Strategy 

All of the USI sites (with the exception of Phoenix, which included all of 
its schools from the beginning) implemented the USI program using a 
scale-up strategy. Cohorts of USI schools were selected, with the focus of 
the reform efforts directed towards these schools. Each year additional 
schools are added until the goal of 100% involvement of the district’s 
schools is reached. The majority of sites planned to implement the 
program systemwide by the fourth year. In Cohorts 93 and 94, 10 out of 
15 sites implemented the program in more than 95% of their schools by 
the fourth year. In Cohort 93, Dallas, Detroit, Miami-Dade, New York, 
and Phoenix implemented the program in at least 95% of their schools, 
and in Cohort 94, Cleveland, Fresno, Memphis, New Orleans, and 
Philadelphia implemented the program in 99% or more schools by the 
fourth year. 

° Student Tracking 

Tracking is the practice of grouping students by their perceived ability 
and steering them into specific courses. This results in lower aspirations 
and expectations for many students. Thirteen USI sites reported 
that tracking was eliminated in the first year of implementation; they 
accomplished this by eliminating all remedial and “non college- 
preparatory” mathematics and science courses. 

° Graduation Requirements 

Science and mathematics requirements for high school graduation have 
increased in most USI districts as shown in Exhibit III. At the start of 
involvement in the USI program, seven sites required two years, 13 sites 
required three years, and one site required four years of mathematics 
for graduation. In SY 98-99, only one site still required two years of 
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mathematics, whereas 17 sites required three years and three sites 
required four years for graduation. In all, eight sites increased their 
expectations for student mastery of mathematics. 

For science, the findings are parallel. At the start of involvement in the 
USI program, two sites required one year, ten sites required two years, 
eight sites required three years, and one site required four years of 
science for graduation. In SY 98-99, seven sites still required two years of 
science, while 13 sites now required three years and two sites required 
four years for graduation. A total of seven sites raised their graduation 
requirements for science. 

It is noteworthy that 12 sites exceed state graduation requirements for 
mathematics and eight sites exceed state requirements for science. 



Exhibit III: High School Graduation Requirement Changes in Mathematics & Science 

From Baseline to SY 1998-99 





Mathematics 
(Number of Years) 

12 3 4 


Science 
(Number of Years) 

1 2 3 


4 


Algebra 1 
& Geometry 
Required 


Laboratory 

Science 

Required 


District Requirements 
Higher than State Requirements 
Math Science 




Baltimore 


□ 


□ 


Yes 


Yes 


No 


No 




Chicago 


O KD 


o 

o 


Yes 


Yes 


Yes 


Yes 




Dallas 


O KC 






Alg.1 


No 


Yes 


Yes 








£ 

£ 


Detroit 


O — KD 


o — «o 




No 


No 


N/A 


N/A 


o 

-c 


El Paso 


□ 


□ 




Yes 


Yes 


No 


Yes 




Miami-Dade 


□ 


□ 


Yes 


Yes 


No 


No 




New York 


□ 


O KD 


Yes* 


Yes* 


Yes 


No 




Phoenix 


O KD 


□ 


No 


No 


Yes 


No 




Cleveland 


O — KD 


O — O 


No 


No 


Yes 


Yes 




Columbus 


O — KD 


O KD 




No 


No 


Yes 


Yes 


3n 


Fresno 


O — O 


O O 




Yes 


Yes 


Yes 


Yes 


£ 

Q 

■s 


Los Angeles 


□ 


□ 


No 


No 


Yes 


No 


<3 


Memphis 


□ 


□ 




Alg.l 


Yes 


No 


No 




New Orleans 


O K3 


□ 




Alg.1 


Yes 


Yes 


No 




Philadelphia 


□ 


□ 


No 


No 


No 


No 




Milwaukee 


□ 


□ 


No 


No 


No 


No 


& 

£ 


St. Louis 


□ 


□ 


No 


No 


Yes 


Yes 


o 

3 


San Antonio 


□ 


□ 




Alg.1 


Yes 


No 


No 




San Diego 


□ 


□ 




Yes 


Yes 


Yes 


No 


tv. 

On 

£ 


Atlanta 


□ 


□ 




No 


No 


No 


No 


3 


Jacksonville 


□ 




□ 


Yes 


Yes 


Yes 


Yes 



"*■ Indicates changes in the number of years required for graduation from baseline to SY 1998-99 
□ Indicates no change in the number of years required for graduation from baseline to SY 1998-99 
* Regents Level Courses 
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These more challenging graduation requirements have not resulted in a 
decline in graduation rates. In fact, graduation rates have increased 
slightly between the baseline year and SY 1997-98, from 84.2% to 84.9% 
according to KIDS- 1999. 

0 Safety-Wet Programs 

The safety net for those students experiencing difficulty with challenging 
mathematics and science courses has been broadened with the 
implementation of the USI. Examples of safety-net programs include 
tutoring, after school enrichment programs, summer school, parent- 
teacher conferences, Individual Education Plans for special education 
students, and bilingual programs for Limited English Proficient students. 

0 Special Education and Limited English Proficient Students 

Most of the USI sites practice inclusion — mainstreaming special 
education students into “regular” classrooms whenever possible. 

A basic premise of the USI program is that all students can learn. 

Therefore, with safety-net programs in place, special education 
and Limited English Proficient students are held to the same high 
standards as regular students. 

PARTNERSHIPS AND LEADERSHIP: BROAD-BASED SUPPORT 
Partnerships 

Every USI site reported numerous strong partnerships. The infrastructure of 
partnerships built around each USI site provided strong support for systemic changes 
based on their unique capacity, resources, and experiences. The partners and school 
districts interacted as part of a unified effort to promote and support large-scale 
education reform. These partnerships created avenues to connect all interested 
community members with the school system. Exhibit IV visualizes the general 
infrastructure of USI partnerships by identifying partner group entities with their typical 
roles and services. These six groups are: higher education; corporations and 
foundations; informal institutions; parents and communities; research centers and 
laboratories; and other fed eral/st at e/local programs. 

Higher education partners - over 130 universities and colleges were reported in KIDS- 
1999 — supported a high quality K-16 mathematics and science continuum targeted to all 
K-12 students and teachers. To achieve this goal, various programs and activities have 
been designed and implemented. Examples include: teacher preservice and inservice 
programs; professional development; teacher graduate degree programs; curriculum 
development and enhancement; mentoring and co-teaching; internships; tutoring 
students, and various student enrichment programs. 

For example, Dallas USI higher education partners — 12 state and private universities and 
colleges - provided assistance with professional development for preservice teachers, 
recertification of current teachers, and provided custom-designed coursework that meet 
state and district curriculum frameworks. For the Memphis USI, the University of 
Tennessee, Christian Brothers University, University of Memphis, and LeMoyne-Owen 
College provided student summer enrichment activities in science, technology and 
algebra, tutorial programs, algebra study labs, and Saturday Academies to support 
achievement in algebra, geometry, precalculus, physical science, biology, chemistry and 
physics. Other Saturday programs were aimed at science and mathematics activities for 
elementary school students. The universities also collaborated with the Memphis USI to 
provide professional development. 




Informal institutions - more than 1 10 institutions were reported in KIDS- 1999 - offered 
rich learning and teaching environments for hands-on, inquiry-based mathematics and 
science education. For example, Baltimore USI students and teachers took advantage of 
the Consortium of Informal Science Institutions including the Baltimore Zoo (over 
31,000 student visits, 90 students attended summer camp, 28 science teachers 
participated in workshops during SY 97-98), Columbus Center (1,400 students and 32 
teachers involved), National Aquarium, Maryland Science Center (26,050 students, 

70 adults, and 674 teachers reached), Irvine Nature Center (8 schools involved in science 
project), and the Chesapeake Bay Foundation (500 students and 135 teachers involved). 
The consortium facilitated communication and resource sharing among member 
institutions, aligned program offerings to the Baltimore City Public School System 
(BCPSS) and Maryland state content standards, and made an effort to involve under- 
represented minority families and community members in supplemental science, 
mathematics, and technology activities. 

Partnerships were formed with'more than 125 corporations, foundations, research 
centers, and laboratories. These partnerships played a major role in educational 
technology training, and funding support. For example, in the School District 
of Philadelphia, numerous organizations (Annenburg Institute for School Reform, 

The Association for the Advancement of Science, Franklin Institute, Princeton Plasma 
Physics Laboratory, Texas Instruments, Inc., Voyager Expanded Learning, Inc., Jason 
Foundation, etc.) provided science and mathematics professional development regarding 
research-based evaluation and assessment, inquiry-based teaching, and curriculum 
content and materials. Two additional NSF-funded initiatives, the Collaborative for 



Exhibit IV: USI Partnership Infrastructure to Support Learning & Teaching 
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Excellence in Teacher Preparation (CETP) and Sisters in Science, were closely 
coordinated to develop new science and mathematics courses, provide after-school 
programs for students, classroom assistance for teachers, and model professional 
development programs. 

Most USI sites reported forming parent and community partnerships. The Detroit USI 
highlighted an exemplary parent and community partnership with a priority on family- 
based mathematics and science education. Family Mathematics and Science programs 
strengthened the link between home and school considered essential to the realization of 
the goal of each student reaching his or her maximum potential. Detroit’s Michigan 
Statewide Systemic Initiative (MSSI) Focus District Grant made significant contributions 
to the family programs. The Center for Learning Technologies in Urban Schools 
(LeTUS) is a partnership among Detroit Public Schools, Chicago Public Schools, The 
University of Michigan, and Northwestern University. The focus of this group is to 
infuse educational technology in the middle school curriculum. 

Leadership 

When the school district’s superintendent supports the USI project and has continuity 
of employment, rates of student participation in mathematics and science courses are 
enhanced. An effective project director and staff can sustain the program even when 
there is turnover in the superintendent’s office. To assess the effects of leadership, 
our evaluative study team assigned rubric scores to each site based on their assessment 
of 1) the continuity of the superintendent and his or her support of the USI program, 2) 
whether the USI project director directly reports to the superintendent, and 3) whether 
there are sufficient numbers of USI staff to implement the program. The latter two 
items compose an index of the effectiveness of the USI project director and staff. When 
statistical methods were applied in the initial analysis to assess the impacts of these 
variables on the percent change in the combined rate of mathematics and science course 
enrollment, the effect of a stable and supportive superintendent becomes apparent. This 
effect is more than twice as large as that of the index that combines the effects of the 
USI project director’s reporting relationship with the superintendent and the size of the 
USI staff. 1 




The standardized regression coefficients that assess the impacts of the variables are: 0.49 for Stable and Supportive 
Superintendent, and 0.21 for the index based on the effectiveness of the USI Project Director and Staff. Both effects are 
large and statistically significant (p<.001). 



COIMWERGEIMCE OF EDUCATIONAL RESOURCES 

The funding provided by the USI program is intended to be a catalyst for the 
mathematics and science education reform effort. While comprising only a small 
amount of the total school district budget, the USI money is discretionary. According to 
the Core Data Element Findings (CDE-1999), each site’s USI funds have leveraged 
in-kind contributions averaging $1 1 million during fiscal year 1999 from various sources, 
such as Federal, state, local, higher education, business, industries, and private 
foundations. 

USI resources are serving a lead role in 
changing the infrastructure within the school 
districts. During site visits and interviews 
conducted in years 1999 and 2000, the study 
team observed collaboration among USI, 

Title I, Eisenhower, other federal initiatives, 
and state and local funding to create a source 
of revenue for professional development, 
curriculum materials, and special programs. 

Twenty USI sites reported that they leveraged 
a total of $547 million in additional funding 
to support SI activities during FY 1999, 

Exhibit V illustrates the percentage distri- 
bution of average USI and total budgeted 
funds (from all sources) directed towards 
reforming mathematics and science 
education during fiscal year 1999. In 
general, USI expenditures parallel those 
from all other sources. Professional development activities are the largest budget item 
for both USI (49% of total USI funds) and total SI funds (34% of total SI funds). 



Additional Funding to Support 
SI Activities in FY 1999 



Source ? A ™“ nt 

in Million 


Other District/School Funds 


166 


State 


148 


Title 1 


134 


Eisenhower 


24 


Department of Education 


14 


D.O.E. Goals 2000 


8 


Other NSF Funds 


6 


Corporate 


5 


Foundation Grants 


2 


Other 


40 


Total Leveraged 


$547 Million 



Source: CDE-1999 



Exhibit V: Average Percentage of USI Budget Allocation 
in Comparison to District-Wide SI Budget in FY 1999* 



Student Tech & 




( % ) Indicates District-wide percentage 

* Refers to all funds (from any source) that are being directed toward reforming mathematics and 
science in the Systemic Initiative during FY 1999. 

Source: CDE-1999 
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PROFESSIONAL DEWELOPIMIEIMT 

Professional development is a major strategy used by all USI sites and supported by a 
variety of district policies. The primary aims of professional development activities are 
to improve teachers 5 content knowledge, pedagogic skills for inquiry-based teaching, 
and methods of assessing student learning. An average of 49% of USI funds were spent 
on professional development activities across all of the USI sites. 

During the 98-99 school year, 75% (1 18,399) of teachers working in USI schools attended 
some form of professional development. Middle school mathematics (84%) and science 
(81%) teachers had the highest rates of participat ion in professional development 
activities as shown in Exhibit VI. 



Exhibit VI: Percent of Teachers who Participated in 
Professional Development by School Level SY 1 998-99 
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Source: KIDS-1999 



Each district delivers professional development in its own way, but there are 
commonalties found across all 21 USI sites. Science and mathematics professional 
development focuses on mastery of content knowledge and standards, pedagogy, 
and assessment strategies. Other topics covered include the use of specific curriculum 
packages the district has adopted (i.e., Connected Math Project, Full Option Science 
System), technology, leadership training, and new teacher orientation. The professional 
development activities are conducted during summer institutes, Saturday training 
sessions, after school programs, and during the school day with classroom coverage by 
substitutes. In addition, site-based and classroom professional development is provided 
by lead teachers. In most sites, the USI leads the planning and implementation of 
professional development in collaboration with other federal grants (i.e., Title I, 
Eisenhower) and the regular school budget. Universities, colleges and other institutions 
(i.e., science centers and museums) support professional development through in-kind 
contributions of instructors, and/or materials and through tuition reductions or waivers. 

As mentioned above, another common approach to professional development is the 
identification and training of lead teachers (i.e., Resource Teachers in Los Angeles, Key 
Teachers in San Diego, and Model Teacher Leaders in Atlanta) who serve as mentors, 
models, peer coaches, and curriculum support resources for teachers in the schools in 
which they work. These teachers focus on developing ways to support classroom change 
at the individual school level. The use of lead teachers was reported by 14 of the 19 
sites. These teachers are supported by Teacher Academies or Leadership Institutes in 
five US districts. 
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Principal Institutes were established in seven sites - Baltimore, Fresno, Memphis, 
Miami, New York City, Phoenix, and San Diego - to assist school based administrators to 
mentor teachers, and to support their efforts to adapt new curriculum content and 
pedagogy. These institutes also directly addressed the issues of standardized testing, 
use of results to evaluate learning and teacher performance, and over-all school 
improvement plans. 

To explore the impact of professional development (PD) on teaching practices and 
student achievement, our USI evaluative study team conducted a Survey of Enacted 
Curriculum in four Cohort 93 USI sites (Baltimore, Dallas, Detroit, and Phoenix) in 
1999 and four Cohort 94 sites (Columbus, Fresno, Memphis, and Philadelphia) in 2000. 
The survey provides an analysis of urban school classroom practices in mathematics and 
science, 

in relation to expectations for practice as outlined in state/district content standards, 
reform initiatives, and standards-based assessment. The third evaluative study 
monograph, entitled Survey Results of Urban School Classroom Practices in Mathematics and 
Science: 2000 Report, reports the summary of findings based on the responses from a total 
of 317 teachers. A few highlights are: 

• About 80 to 90% of USI teachers were actively involved in professional 
development, which focused on content standards, in-depth study of 
content, curriculum implementation, multiple strategies for assessment, 
and new methods of teaching. Teachers report that as a result of 
professional development they are using and applying new methods and 
standards in classrooms (average 2.3 on scale of 1 to 3: scale 1 = had 
little or no impact, 2 = trying to use, and 3 = changed my teaching 
practice). 

• In science, teachers with High PD (more than 16 hours in content and 
pedagogy) report greater use of multiple assessments than their 
counterparts with Low levels of professional development (Low PD), 
especially at the elementary level. 

• For science and mathematics, state and district frameworks or standards 
have the greatest positive influence on curriculum, as well as national 
standards. Differences between High PD and Low PD teachers were 
not notable. 



“Structure of the day has to be changed to accommodate those children. The problem is that 
some teachers don't have that training. With USI training and the way it has been set up, 
it allows teachers and students to interact a lot and makes it so much easier. One of the 
goals of our school system is to align the curriculum with state goals. USI does a good job 
of doing that. 33 

— A Principal in a USI school 




t 
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USI Implementation School Year 



TIMELINE OF OSD DMPDJEMENTATOON 

Each of the USI sites has its own unique implementation plan for changing the 
educational infrastructure to support improved student achievement in mathematics 
and science. Exhibit VII presents an example from Miami-Dade illustrating the 
milestones of its USI implementation from baseline to SY 98-99, along with student 
enrollment trends in mathematics and science gate-keeping and higher-level courses: 

Exhibit VII: Milestones of USI Implementation in Miami-Dade 
(Major Policy Changes vs. Enrollment Trends in Mathematics and 
Science Gate-Keeping and Higher-Level Courses) 



98-99 



Professional development (PD) as part of school improvement plans 
Para-professionals included in PD activities 



97-98 



1 Remedial instruction provided for students not passing performance standards 
1 Parent conference required for students not meeting district mandated 
performance goals Q 

1 Principal Institute implemented 



96-97 



1 Mathematics and science curricula revised to align with state and 
national standards 

1 Elementary teachers' PD requirement - 1 hr/ wk 
1 Secondary teachers' PD requirement - 1 hr/ month 



95-96 



94-95 



* Use of test scores for course enrollment eliminated 

► All elementary schools implement standards-based curriculum 

► Middle school students must pass 3 years each of mathematics and science to 
be promoted to high school 



// 

// 



1 Basic level mathematics and science courses eliminated 
1 State standards adopted 

- Competency based curriculum developed 

- Algebra I required for graduation 
■ Teacher consultants appointed 

- Baseline Year // 



93-94 



6 

51 ,493 



i 



40,000 



50,000 



60,000 



70,000 



80,000 



Number of Students Enrolled in Mathematics and Science Gate-Keeping and Higher-Level Courses 



“What we can see now as we look back is that we have moved agenda around the curriculum , 
around how we do professional development. But I think , more meaningful ly 3 we have moved 
agenda to open up opportunity to kids." 

- A USI Project Director 



STUDENT ACCOMPUSMMENTS ON (MATHEMATICS AND SCBENCE 

For the 21 USI sites, the Key Indicator Data System (KIDS) has compiled outcome data 
that monitors student progress from the USI baseline year to SY 98-99. KIDS is a 
quantitative (K-l) and qualitative (K-2) annual data collection instrument designed and 
implemented as a component of the USI evaluative study being conducted by Systemic 
Research, Inc. Exploratory trend analysis indicate positive impacts of the USI program 
on students’ mathematics and science achievement. The student participation and 
outcome measures include enrollment in and completion of mathematics and science 
gate-keeping and higher-level courses, state and district assessment test results, 
and results from Advanced Placement (AP) tests, the Scholastic Aptitude Test (SAT), 
and the American College Test (ACT). 

OIMTOCWED IMATHEIMATOCS AMD SCIENCE GATE-DCEEPBNG AND 
H0GHEF5-LEWEIL. COURSE ENROLLMENT AMD COMPLETION 

Urban school students show progress in enrollment and completion of mathematics 
and science gate-keeping and higher-level courses. These courses include Algebra I or 
above in eighth grade; and Algebra II, Geometry, Calculus, Integrated Mathematics, 
Biology, Chemistry, Physics, and Integrated Science in grades 9-12. In general, cohorts 
with longer durations in the USI program have larger increases in enrollment in 
mathematics and science than cohorts involved in the program for shorter periods of 
time. Enrollment increases in mathematics and science are far greater than 
demographic shifts in these districts. 

For each cohort, the graphs of Exhibit VIII portray the growth in 9-1 2th grade enrollment 
in mathematics and science gate-keeping and higher-level courses. The percentages 
above the enrollment numbers indicate the percentage of all students who are enrolled in 
these higher-level mathematics and science courses. For example; in Cohort 93, 158,673 
students, or 30% of the total number of 9- 12th grade students, were enrolled in gate- 
keeping and higher-level mathematics courses during SY 93-94. This number increased 
to 253,458 students which is 46% of that year’s total number of 9- 1 2th grade students. 

The longest sustaining cohort (Cohort 93) indicates an enrollment increase of 16 % 
points from the baseline to SY 98-99 in mathematics and a corresponding 22 % points 
increase in science. The cohorts with later start dates indicate enrollment increases in 
mathematics of +6 % points, +4 % points, and +2 % points in Cohorts 94, 95, and 97, 
respectively. The enrollment in science courses increased in Cohorts 94 and 95 from the 
baseline year to SY 98-99, +9 % points and + 7 % points, respectively. Cohort 97 
students experienced a -2 % points decrease in science enrollment. 

Urban middle school students, including underrepresented minority students, have also 
experienced increases in Algebra 1 or above in 8th grade course enrollment, as shown 
in Exhibit IX. In the four Cohort 93 sites with available data (Chicago, Dallas, Detroit, 
and Miami-Dade) Black students increased their enrollment in Algebra 1 or above in 8th 
grade from 50 enrollees per 1,000 in SY 93-94 to 79 per 1,000 in SY 98-99, a 58% increase. 
Hispanic students had an enrollment increase of 59% from 87 per 1,000 to 138 per 1,000, 
and White students increased their enrollment rate 26% from 170 to 214 per 1,000. 

Algebra 1 or above 8th grade enrollment trends in Cohort 94 sites (Cleveland, 

Columbus, Fresno, Los Angeles, Memphis, and Philadelphia) have shown mixed results. 
Black students exhibited a remarkable 82% enrollment rate increase from 125 enrollees 
per 1,000 in SY 94-95 to 228 per 1,000 in SY 98-99, compared to a 10% increase 
(from 242 to 265 per 1,000) of White students, and a 22% decrease (from 186 to 145 
per 1,000) of Hispanic students. 
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Exhibit VIII: Number of Students Enrolled in Grade 9-12 
Gate-Keeping and Higher-Level Mathematics and Science Courses 
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Cohort 93 ( 6 ): 

Baltimore 

Chicago 

Dallas 

Detroit 

Miami-Dade 

New York 



93-94 94-95 95-96 96-97 

School Year 



97-98 



98-99 



■O" Science Enrollment 
Math Enrollment 



Example: During SY 1993-94 in Cohort 93, there were 158,673 students enrolled in mathematics 
gate-keeping and higher-level courses, which is 30% of the total grade 9-12 student enrollment. 



* Math: Algebra II, Geometry, Calculus, Integrated Math ll-IV. Science: Biology I, Chemistry I, Physics I, and 
Integrated Science l-lll. ( % ) Percent of Total Grade 9-1 2 Enrolled. 



Exhibit IX: Number per 1 ,000 of Total 8th Graders Enrolled in Algebra I or Above by Race/Ethnicity 
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+ 1 Cohort 93: Data includes Chicago, Dallas, Detroit, and Miami-Dade 

* 2 Cohort 94: Data includes Cleveland, Columbus, Fresno, Los Angeles, Memphis, and Philadelphia 











The completion rates of students taking gate-keeping and higher-level mathematics and 
science courses are smaller than the enrollment increases from the baseline to SY 98-99 
in most cases. The difference between enrollment and completion rates is expected due 
to the large increases in total number of students enrolling in these challenging courses. 
For example, the mathematics enrollment rate increased 62% and the completion rate 
increased by 50% for Cohort 93 students; the enrollment rate increased 25% and the 
completion rate 9% for Cohort 94; and the enrollment rate increased 18% and the 
completion rate 16% for Cohort 95. The mathematics enrollment rate for Cohort 97 
students increased 8% and the completion rate 10%. The science completion rates 
exhibited a similar pattern. 

REDUCING ENROLLMENT DOSPARDTOES 

The gate-keeping and higher-level mathematics and science course enrollment rates 
have increased for all high school students in the USI sites. The rate of increase for 
underrepresented minority students (Black and Hispanic) has outpaced that of White 
students, thereby reducing the enrollment disparity. 

Exhibits X & XI illustrate increases in course enrollment and decreases in enrollment 
disparities from SY 93-94 to SY 98-99 for five Cohort 93 sites - Chicago, Dallas, Detroit, 
Miami-Dade, and New York. Exhibit X compares Black-to-White and Hispanic-to-White 
enrollment rates for 9-1 2th grade gate-keeping and higher-level mathematics courses. 
The enrollment figures are presented as the number enrolled per 1,000 9- 12th 
grade students. In SY 93-94 the Black-to-White enrollment disparity in grade 9-12 
mathematics gate-keeping and higher-level courses was 94 students per 1,000. By SY 
98-99, this disparity had narrowed to 74 per 1,000 (a 21% decrease) as enrollment rates 
increased significantly for all students. Similar trends are seen for Hispanic-to-White 
disparities. 

As shown in Exhibit XI, from SY 93-94 to SY 98-99 Black-to-White and Hispanic-to- 
White enrollment disparities for 9-1 2th grade gate-keeping and higher-level science 
courses have also been narrowed. The Black-to-White disparity in science enrollment 
has been reduced 27.9%, from 183 to 132 science enrollees per 1,000; the Hispanic-to- 
White disparity has been reduced 38.7%, from 253 to 155. 



“With the inquiry method for science , one of the hardest things for teachers is to be 
able to have the children lead and take over Once you see them doing that , it is 
awesome-how they are able to go hypothesize, get all the equipment, decide what they 
want to find out, do the experiment and report back to the class 

— A teacher in a USI school 
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Exhibit X: Reduced Disparity and Increased Enrollment in Grade 9-12 Mathematics Gate-Keeping 
and Higher-Level Courses*: Number of Students Enrolled in Mathematics per 1 ,000 Students 
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Example: In the SY 93-94, 69,639 Black students enrolled in mathematics gate-keeping and higher- 
level courses, which represents a rate of 291 per one thousand students. In SY 98-99, the number of 
black students enrolled in math gate-keeping courses increased to 108,578 (455 per 1 ,000) 



* Includes Algebra II, Geometry, Calculus and Integrated Math ll-IV: Data includes Chicago, Dallas, 
Miami-Dade, and New York 



Exhibit XI: Reduced Disparity and Increased Enrollment in Grade 9-12 Science Gate-Keeping 
and Higher-Level Courses*: Number of Students Enrolled in Science per 1 ,000 Students 
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* Includes Biology I, Chemistry I, Physics I, Integrated Science l-lll: Data includes Chicago, Dallas, Detroit, 
Miami-Dade, and New York 



“Our plan is about the issue around equity in our district . It was real clear The audit said 
that the most significant issue we have is the equity of results around student achievement , 
for students between poor and non-poor and between black and white in our district 33 

- A Superintendent in a USI school district 













HONKING QUAUTATOWE ONDOCATORS TO STU1DENT PAOOTICOPATOON: 
A PRELIMINARY APPROACH 

Do the policies fostered by the USI program impact students’ achievement in 
mathematics and science course enrollment, as shown in Exhibit VIII? To explore the 
causal relationships between policies and student outcomes, our evaluative study team 
developed a template to assign scores to 46 qualitative indicators (policy rubrics) that 
reflect the drivers of systemic reform. An initial analysis assesses the influence often 
policy rubrics focusing on creating high expectations, promoting equal access, and 
policies impacting higher-level mathematics and science course enrollment. 

Based on the scored rubrics, a statistical method (factor analysis) combined ten policy 
rubrics into three independent factors: High Expectations (summer school, strict 
promotion policies, rigorous graduation requirements, and tutoring or Saturday 
Academy); Equal Access Because All Can Learn (more upper-level courses, high 
attendance policy, and elimination of tracking); and Sensitivity to Different Learning 
Styles but with Universal Standards (targeted programs for underrepresented minorities, 
strategies that address the needs of special populations, and rigorous graduation 
requirements). 

When these three factors are related statistically to the improvement rate of 
mathematics and science course enrollment, the result is that high expectations and 
equal access policies have the greatest impact on students’ improvement in mathematics 
and science course enrollment. Sensitivity to Different Learning Styles, while still 
having a positive effect, makes less of a difference overall. 2 

Our study will continue to explore the impact of the other 36 qualitative indicators that 
include curriculum and instruction, teacher qualifications, professional development 
and practices, leadership, partnerships, assessment, and accountability. 

ASSESSMENT TEST RESULTS 

Exhibit XIII illustrates trends in assessment test results for 8th grade students in 
mathematics, comparing White students to students that comprise the largest 
underrepresented minority group in a given district. The 16 USI districts that are 
represented in the graph administered to their students the same 8th grade test over two 
or more years. The graph depicts changes in the Minority-to-White gap between the 
baseline (or earliest available) year and the most recent available year. These data 
describe changes within a site. Differences between sites in their percentages of 
students who passed a test are affected by the definition of passing used; consequently, 
comparisons of the magnitudes of these percentages from one site to another are not 
meaningful. The length of a bar indicates the gap in the passing rate between White 
students and the largest underrepresented minority group in a given district. 

Each district has its own test and performance benchmarks as indicated in Exhibit XII. 

In a majority of these sites the passing rate improved in the final year compared with the 
earlier year, both for Whites (15 of 16 sites experienced gains) and for the largest 
minority group (15 of 16 sites experienced gains). 



The standardized regression coefficients that assess the impacts of each factor are: 0.230 for High Expectations, 
0.168 for Equal Access, and 0.067 for Sensitivity to Different Learning Styles. 



ERIC 
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In Cohort 93, the longest sustaining cohort, five out of the seven sites have gap 
reductions. The non-weighted average gap reduction across these seven sites is -6.3 % 
points, which is greater than the reductions in the other cohorts. In Cohort 94, two of 
the sites reported gap reductions; the non-weighted average gap narrowed slightly by 
-2.4 % points. In Cohort 95, one of the four sites reported a reduction; the average 
change is -3.5 % points. Additionally, the five sites that were predominantly Hispanic 
had an average reduction in the gap of -8.0 % points, whereas the nine sites that were 
predominantly Black had a slight average gap increase of + 1.0 % points. 

Exhibit XIV presents parallel data on the passing rates for 8th grade assessment tests in 
science for 15 USI districts. Overall, in 14 out of 15 sites, the passing rate is higher in 
the final year of the comparison than in the earlier year for all students. In Cohort 93, 
three of the six sites have gap reductions; the non-weighted average gap reduction is 
-3.9 % points. In Cohort 94, one of the sites reports a gap reduction; the non-weighted 
average increase in the gaps is + 1.0 % points. In Cohort 95, all four sites report 
reductions; their non-weighted average is -4.8 % points. Once again, the five sites that 
were predominantly Hispanic had the larger non-weighted average reduction in the gap: 
-5.6 % points compared to a non-weighted average gap increase of +0.3 % points in the 
nine predominantly Black sites. 

Exhibit XII: 8th Grade Mathematics and Science 

Assessment Tests and Performance Benchmarks 
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* Shaded Green areas represent benchmark categories used to determine passing rates in Exhibit XIII & XIV. 
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Exhibit XIII: 8th Grade Mathematics Assessment Test Results 
Gap Between Passing Rates of Largest Minority Group and White Students 
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Example: In El Paso, 43% of the largest minority group (Hispanic) and 73% of Whites passed the TAAS 
in 1 993-94; in 1 997-98, 78% of Hispanics and 91 % of Whites passed the TAAS. Thus the gap between 
Hispanics and Whites decreased from 30 to 13 percentage points 



+1 9th Grade data 

Data are not available for Atlanta, Fresno, Jacksonville, Philadelphia, and Phoenix. Blacks 
comprise the largest minority group in Baltimore, Chicago, Cleveland, Columbus, Detroit, 
Memphis, Milwaukee, New Orleans, and St. Louis. Hispanics comprise the largest minority 
group in El Paso, Los Angeles, Miami, San Antonio, and San Diego. In Dallas and New York, 
both Blacks and Hispanics comprise over 30% of the total student population; the average 
passing rate between Blacks and Hispanics is provided. 





Exhibit XIV: 8th Grade Science Assessment Test Results 
Gap Between Passing Rates of Largest Minority Group and White Students 
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Example: In El Paso, 53% of the largest minority group (Hispanic) and 83% of Whites passed the TAAS 
in 1994-95; in 1997-98, 80% of Hispanics and 95% of Whites passed the TAAS. Thus the gap between 
Hispanics and Whites decreased from 30 to 1 5 percentage points 



* 1 9th Grade data * 2 7th Grade data 

Data are not available for Atlanta, Fresno, Jacksonville, New York, Philadelphia, and Phoenix. 
Blacks comprise the largest minority group in Baltimore, Chicago, Cleveland, Columbus, Detroit, 
Memphis, Milwaukee, New Orleans, and St. Louis. Hispanics comprise the largest minority 
group in El Paso, Los Angeles, Miami, San Antonio, and San Diego. In Dallas, both Blacks and 
Hispanics comprise over 30% of the total student population; the average passing rate of Blacks 
and Hispanics is provided. 
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COLLEGE ENTRANCE EXAMINATIONS 

The taking of college entrance examinations, Scholastic Assessment Test (SAT), 
American College Testing (ACT), or Advanced Placement (AP), is an indication of the 
students 5 intent to pursue their education beyond high school. Students in Cohort 93, 
the longest sustaining USI cohort, show the greatest increase in test-taking rates 
compared to the remaining three cohorts. 

° AP. Mathematics and Science Tests 

The sites in Cohort 93 are characterized by greater increases in their 
rates of AP test taking by 1 1-1 2th grade students. The graphs of Exhibit 
XV illustrate the changes in these rates for all cohorts along with the 
national trend. For mathematics, Cohort 93 students (four years of USI 
implementation) increased their test-taking rate by seven more test 
takers per 1 ,000, compared to an increase of just 1.7 per 1 ,000 for Cohort 
94 students, and a decrease of -.8 per 1,000 for students in Cohort 95 
sites. By SY 97-98 the test-taking rate for Cohort 93 students exceeded 
the national rate by 2.1 test takers per 1,000. 

For science, Cohort 93 students increased their test-taking rate by 
10.3 per 1,000, compared to an increase of 2.9 per 1,000 for Cohort 94 
students, and to an increase of about 1 per 1,000 for Cohort 95 students. 

By SY 97-98 the rates for students in Cohort 93 and Cohort 94 exceeded 
the national rate by 1.9 and 3.8 test takers per 1,000, respectively. 

Exhibit XV: Trends in Mathematics and Science Advanced Placement 
Test Takers per 1 ,000 1 1th and 1 2th Grade Students* 



Mathematics AP 
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Cohort 95 



Science AP 
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* AP Math tests: Calculus AB and BC. AP Science tests: Biology, Chemistry, and Physics 
B and C. Based on available AP data for Cohort 93 (Baltimore, Chicago, Dallas, Detroit, 
El Paso, Miami-Dade, New York, and Phoenix), Cohort 94 (Cleveland, Columbus, Fresno, 
Los Angeles, Memphis, New Orleans, and Philadelphia) and Cohort 95 (Milwaukee, 

St. Louis, San Antonio, and San Diego). 
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° SAT and ACT 

Most colleges and universities, as part of their application procedure, 
require their applicants to take the SAT or the ACT Exhibit XV 
illustrates the test-taking rates of those sites whose students 
predominantly take the SAT; Exhibit XVI shows those sites whose 
students predominantly take the ACT Both graphs indicate that Cohort 
93 sites have experienced increased rates of SAT and ACT test-taking, 
compared to the nation, and in most cases, to the later cohorts. 

Exhibit XVI depicts how SAT test-taking rates in Cohort 93 increased 
from 223 per 1,000 1 1-1 2th grade students in SY 93-94 to 250 test takers 
per 1,000 in SY 97-98, an increase of 27 per 1,000. The national trend 
experienced little change in the same time period. Cohort 94 students 
had a small decrease of -4 per 1,000 and Cohort 95 students an increase 
of 1 1 test takers per 1,000. 

ACT test-taking trends are illustrated in Exhibit XVII. Cohort 93 
students experienced a small decrease of -6 test takers per 1,000 between 
SY 93-94 and SY 97-98 from 245 to 239 per 1,000, compared to the 
national trend decrease of about -10 test takers per 1,000. However, 
the test-taking rates increased by 8 per 1,000 for Cohort 94 students and 
by 2 per 1,000 for Cohort 95 students, from the baseline to SY 97-98. 

Exhibit XVI: Rate of Taking SAT Test Exhibit XVII: Rate of Taking ACT Test 

per 1 ,000 1 1 th and 1 2th Grade Students* per 1 ,000 1 1 th and 1 2th Grade Students* 



SAT Test 



ACT Test 




School Year School Year 



* SAT sites are: Cohort 93 (Baltimore, * ACT sites are: Cohort 93 (Chicago and 

Dallas, El Paso, Miami-Dade, and New Detroit) Cohort 94 (Cleveland, Columbus, 

York), Cohort 94 (Fresno, Los Angeles, Memphis, and New Orleans) and Cohort 95 

and Philadelphia) and Cohort 95 (Milwaukee and St. Louis). 

(San Diego). 



TALES FROM ED01H1T COTOES - JOURWEYS TO SUCCESS FOR 
ALL STUOEMTS 

During the past two years, our evaluative study team conducted eight USI site visits. 

We met and interviewed numerous educators, administrators, and researchers including 
superintendents, school board members, USI project directors and their staff, 
mathematics and science supervisors/coordinators, professional development 
coordinators, directors for curriculum, instruction, assessment, federal program 
coordinators, core data managers, evaluators, partners, and parents. We also visited 
numerous schools to observe classrooms, and to interview principals, mathematics and 
science teachers, counselors, and students. From the school board to the classroom, 
we saw evidence of the broad impact of the USI. The USI program, partners, and 
stakeholders are creating an infrastructure that supports the success of all students. 

It is not possible to present a comprehensive profile of each site’s activities and progress. 
In their own voice, sites have reported several notable landmarks on their journey to 
success for all students. Each focuses on different facets of systemic reform. 

Baltimore City Public Schools, MD 

With the inception of the USI program, the mathematics and science curriculum in 
Baltimore has been revamped. The curriculum in all grades has been aligned with 
national standards. Content standards have been developed based on Maryland State 
Department of Education Standards, Maryland Learning Outcomes, Maryland High 
School Core Learning and national standards. To support the curriculum, new policies 
have been adopted: elementary school level science instruction time has been lengthened 
from one third to a full year, and graduation requirements have been raised to include 
three years each of mathematics and laboratory science. 

Baltimore City Public Schools (BCPS) have formed a primary partnership with Morgan 
State University (MSU) to administer the USI program. This partnership affords a 
unique opportunity for the district to influence pre-service teacher preparation because 
MSU is a major contributor of BCPS teachers,. Mathematics and science professional 
development in the district is driven by the USI program. Professional development is 
now focused, comprehensive, and integrated. 

Instructional Support Teachers (ISTs) are the bridge between professional development 
and the classroom. ISTs have received over 200 hours of professional development in 
content and pedagogy that enable them to coach, mentor, demonstrate, and co-teach. 

In addition to providing site-based classroom assistance, ISTs present city-wide 
professional development activities and are involved in grade level planning. 

Columbus Public SchooHs, OH 

The USI has adopted a standards-based curriculum aligned with instruction and 
assessment. Written curriculum guides were improved, “best practices” in instruction 
were supported, and student assessment test data to guide teaching and learning were 
used. Reflecting the reality of today’s technology driven society, a course in technology 
and a corresponding internship is a new graduation requirement, beginning with the 
Class of 2003. 

The USI program is the largest provider of mathematics and science professional 
development in the district. At the school level, the Teacher Support Team (TST) 
provided classroom assistance, instructional modeling, peer coaching, team 
collaboration, inservice programs, and data analysis. 

Each elementary and middle school has an instructional leadership team consisting of a 
principal, literacy facilitator, science advocate and mathematics advocate. Along with 





the TSTs, mathematics and science advocates are responsible for on-site professional 
development and teacher assistance at each school. 



Columbus USI has established a number of partnerships with: institutions of higher 
education, informal institutions, commercial providers of professional development, 
and business/parent/community organizations. A major partner is Ohio State University 
(OSU), providing pre-service education, professional development for teachers, course 
design and facilitation, and student centered programs such as the Young Scholars 
Institute. 

Dallas Public Schools, TX 

Policy changes directed toward student improvement in science and mathematics have 
been implemented with the support of the USI program. All secondary students are now 
required to enroll in both a science and mathematics course each year. Graduation 
requirements have been increased to include four years of science and mathematics. 
Algebra IA and IB (non-rigorous Algebra courses) have been eliminated, and seventh 
and eighth grade mathematics courses have been replaced with Pre-Algebra courses. All 
DPS staff are required to participate in 36 hours of professional development per year. 

The USI program supports the adoption of the Texas Essential Knowledge and Skills 
(TEKS) state standards in mathematics and science. School-based mathematics and 
science lead teachers and a leadership team support the implementation of the TEKS 
based curriculum. 

Universities provide professional development for preservice teachers, as well as content 
training and recertification for current teachers. DPS have also engaged in partnerships 
with informal institutions (i.e.. The Science Place, The Museum of Natural History, 

The Arboretum, and the Dallas Zoo) to provide professional development supporting the 
district curriculum framework and national standards. 

DPS encourage community partnerships to provide tutoring and mentoring for its 
students. Tutoring is provided by members of several organizations including The 
National Association of Black Engineers and the Dallas Alliance for Minorities in 
Engineering. Tutors and mentors for both teachers and students are also recruited from 
local universities and businesses. 

Detroit Public Schools, Ml 

The vision of the Detroit Public Schools USI (DUSI) program is that all children will 
reach their maximum potential as they progress through a challenging and rigorous 
sequential science and mathematics K- 12 curriculum. DUSI is the driving force behind 
the science and mathematics reform effort, working with the Michigan Statewide 
Systemic Initiative. 

During the first year of the USI (SY 1994-95) Detroit Public Schools (DPS) implemented 
a new DPS Core Curriculum, based on state curriculum frameworks, the Michigan 
Educational Assessment Program (MEAP), NCTM mathematics standards, and NRC 
science standards. Subsequently, all remedial classes were eliminated in SY 94-95 and 
graduation requirements were increased from two years of mathematics and science to 
three years each in SY 96-97. Constructivist instructional practices are now an integral 
part of core curriculum implementation and professional development offerings. 

Partnerships with external institutions play an important role in systemic reform. A 
priority of DUSI is to involve the community in the schools. There are over 200 
partnerships in individual schools. These partnerships provide mentoring, tutoring, 
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COMCLUSDOMS 

This report presented evidence of noteworthy gains in student achievement, with the 
greatest gains seen in school districts that have participated in the USI program for the 
longest period of time. These advances are accompanied by evidence that urban 
districts are developing the infrastructure to sustain achievement gains, policies that 
encourage enrollment in gate-keeping and higher-level mathematics and science 
courses, strengthened professional development programs, new ways of managing 
partnerships and resources, and data-driven accountability systems. NSF’s Six Drivers 
of Systemic Reform provided a framework for USI implementation, especially focusing 
on standards-based curriculum and instruction, policies supporting all students in high 
quality mathematics and science education, professional development, aligned 
assessment instruments, convergence of resources, leadership, and partnerships. 

Urban students in the 21 USI school districts have dramatically increased their 
enrollment rates in mathematics and science gate-keeping and higher-level courses. 
Underrepresented minority students made even greater enrollment gains than their 
peers during the same period, resulting in reduced enrollment disparities. Assessment 
test results show that USI students have also made gains in mathematics and science 
achievement, while reducing achievement gaps among racial and ethnic groups. 

The increasing numbers of 1 1th and 12th grade students taking college entrance 
examinations indicate more students have aspirations of pursuing post-secondary 
education. By SY 97-98 Cohort 93 students were taking AP mathematics and science 
examinations at a rate higher than the national average - a marked increase since 
1993-94. Cohort 94 students also follow the same trend. SAT and ACT test-taking rates 
show a similar pattern. These trends are most notable in the cohorts with the longest 
participation in the USI program. 

USI programs have invested heavily in professional development believing it to be a key 
lever for improving student outcomes. Seventy-five percent of teachers in the USI sites 
have participated in professional development to enhance their delivery of mathematics 
and science education in their classrooms. Our Enacted Curriculum Survey found that 
teachers with more professional development in standards-based curriculum and 
instruction report teaching practices that are more consistent with state and national 
standards than those with lesser amounts. 

The infrastructure of partnerships built around each USI site provided strong support 
for systemic changes based on their unique capacity, resources, and experiences. The 
partners and school districts interacted as part of a unified effort to promote and 
support large-scale education reform. Partnerships were formed with interested 
community members including higher education institutions, corporations, foundations, 
informal institutions, research centers and laboratories, parents, and other 
federal/state/local programs. 

Education reform is a complex, long-term process that requires simultaneous changes in 
expectations, policies, curriculum, professional development, student support systems, 
assessment, data use, and the allocation of resources. The first five years of the USI 
program have provided a firm foundation of experience and practical wisdom for the 
second phase of reform - the Urban Systemic Program (USP). As students are exposed 
to high expectations, a rigorous and engaging curriculum, and better instructional 
methods, NSF’s investment will continue to result in improved outcomes for all urban 
students. Much has been done; more remains to be accomplished. 
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Systemic Research, Inc. 

150 Kerry Place, Second Floor 
Norwood, MA 02062 
www.systemic.com 



Urban Systemic Initiative (USI) is a K-12 education reform program sponsored by the National Science Foundation. 
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